Layered image rendering

ABSTRACT

A method for creating a rendered scene that is composed of two or more RGB image layers that can be composited at a later time. The resulting composition will appear the same as if it were rendered all at the same time. The elements of a foreground image layer scene are rendered separately over a black background and a white background. The RGB components for each pixel of the foreground image layer scene rendered over the black background are combined with the RGB components for each corresponding pixel of the foreground image layer scene rendered over the white background to form a rendered foreground layer. The foreground layer may be combined with a background image layer in a conventional manner, possibly with additional material added between the background and foreground image layers.

FIELD OF THE INVENTION

[0001] This invention pertains generally to computer graphics and imageprocessing and, more particularly, to systems and methods for generatingand compositing digital images, such as digital images composed of twoor more image layers.

BACKGROUND OF THE INVENTION

[0002] Most computer graphics images are computed all at once, so thatthe rendering program takes care of all computations relating to theoverlap and transparency of objects within the image. There are severalapplications, however, where objects within or layers of an image arerendered separately. In such a case, the compositing techniques used forcombining the objects or layers into a single image are relied upon forthe anti-aliased accumulation of a full image in which the transparencyof foreground layer objects with respect to a background layer isproperly represented.

[0003] Standard digital RGBA images are defined at the pixel level byfour values which define the red, green, and blue color components ofthe pixel, along with an alpha value which is a mixing factor indicatingthe amount of coverage or transparency of the pixel when compositedwith, e.g., overlaid on, a background image. For example, an alpha valuefor a pixel of zero may indicate no coverage or complete transparency,i.e., the pixel is clear, and thus would not in any way obscure abackground layer pixel when composited therewith. An alpha value of onemay mean full coverage or a completely opaque pixel, i.e., the pixelwould completely cover the corresponding pixel of a background layercomposited with the image. Fractional alpha values correspond to partialcoverage by or transparency of the pixel when composited with abackground layer image.

[0004] When compositing computer graphics images of objects or imagelayers which overlap, standard equations are employed to combine thecolor elements of corresponding overlapping pixels in appropriateamounts defined by the alpha values of the pixels (e.g., the alpha valueof the foreground layer pixel). This conventional color combiningprocedure works well in many cases. However, for certain foregroundlayer objects, particularly semi-transparent objects, to be overlaid ona colored background layer, such a compositing procedure employing aformulaic averaging of color contributions made by the foreground layerobject and background layer image will not result in the desiredcomposited image if the foreground layer is rendered in a conventionalmanner. For example, it may be desired to composite a semi-transparentforeground layer object, such as a graphical representation of a whitesnowflake, over a colored background layer. If the semi-transparentforeground layer object (the snowflake) is rendered in RGBA in aconventional manner, the compositing of the foreground layer object (thesnowflake) and background layer in a conventional manner involving anaveraging of the color contributions made by the foreground andbackground layer objects may result in an off-colored object (e.g., agray snowflake) superimposed on a background layer, rather than thedesired semi-transparent object (white snowflake) overlaid on thebackground layer. (A semi-transparent white snowflake over a coloredbackground would be the resulting image if the entire image, foregroundand background layers, were rendered at the same time.) Similar colorartifacts can occur at the edges of opaque foreground layer imageobjects rendered in a conventional manner and overlaid on a backgroundlayer image using conventional compositing techniques.

[0005] What is desired, therefore, is a relatively simple method ofcreating a digital image that is composed of two or more image layersand in which the resulting composition will appear the same as if thescene layers were rendered as a single scene all at the same time.Preferably, such a method permits the creation of image layers which canbe rendered separately and stored to be composited together at a latertime, possibly with additional material added between the rendered imagelayers.

SUMMARY OF THE INVENTION

[0006] The present invention provides a method of creating a renderedscene that is composed of two or more image layers. Background andforeground image layers may be rendered separately, in accordance withthe present invention, and saved to be composited at a later time.Additional material may be added between the rendered background andforeground image layers. The image resulting from composition of theimage layers rendered in accordance with the present invention willappear as if all of the layers forming the image were renderedsimultaneously as a single scene.

[0007] All rendering in accordance with the present invention is done inRGBA. Background and foreground image layers to be composited at a latertime to form a composited image are rendered separately. The elements ofa background image layer may be rendered in a conventional manner inRGBA, and saved for later combination with a foreground image layer.(Alternatively, the background layer may be an RGB only image.)

[0008] Creation of a foreground image layer in accordance with thepresent invention begins with defining a scene of foreground layer imageelements or objects. The background of the foreground image layer thusdefined is erased to full black. The elements of the foreground imagelayer scene are then rendered in a conventional manner over the fullblack background to obtain RGB components for each pixel of theforeground image layer scene rendered over black. The background of theforeground image layer scene is then erased to full white. The elementsof the foreground image layer scene are then rendered in a conventionalmanner over the white background to obtain RGB components for each pixelof the foreground image layer scene rendered over white. The RGBcomponents for each pixel of the foreground image layer scene renderedover black are combined with the RGB components for each correspondingpixel of the foreground image layer scene rendered over white to formthe rendered foreground layer image. This rendered foreground layerimage may be composited with the saved background image layer, and/orother image layers, in a conventional manner, to form a composited scenewhich will appear the same as if the scene were rendered all at the sametime.

[0009] The step of combining the RGB components for each pixel of theforeground image layer scene rendered over black with the RGB componentsfor each corresponding pixel of the foreground image layer scenerendered over white to form the foreground image layer may employ thefollowing procedure in accordance with the present invention to obtainRGBA components for each pixel of the thus rendered foreground layerimage. An alpha value for each pixel of the foreground layer image isdetermined as one plus the value of a color component of thecorresponding pixel from the foreground image layer scene rendered overblack minus the value of the same color component of the correspondingpixel from the foreground image layer scene rendered over white. Forexample, an alpha value for each pixel of the foreground layer image tobe rendered may be determined as one plus the value of the red componentof the corresponding pixel from the foreground image layer scenerendered over black minus the value of the red component of thecorresponding pixel of the foreground image layer scene rendered overwhite. If the resulting alpha value for the pixel of the foregroundlayer image scene to be rendered equals zero, all of the RGB colorcomponent values of the pixel are also set to zero. Otherwise, the RGBcolor component values of the pixel of the foreground layer image scenebeing rendered are set to the corresponding color component values ofthe corresponding pixel from the foreground image layer scene renderedover black divided by the alpha value for the pixel. The resulting RGBAforeground layer image is saved, and may be combined with the RGBA (orRGB) background layer previously rendered, using conventionalcompositing techniques, to create a composited image scene.

[0010] The present invention may thus be employed in combination withconventional RGBA rendering techniques and programs to derive colorcomponents and alpha values for an RGBA foreground layer image to becomposited with a background layer image. An RGBA foreground layer imagerendered in accordance with the present invention may be composited withany background layer scene, and will retain the desired characteristicsof the foreground layer image without blending of the colors of theforeground layer scene with the background layer image. Additional imagelayers, e.g., an image layer positioned between the foreground andbackground layers, may be composited to form a multi-layer scene. Thedesirable visual qualities of the foreground image layer rendered inaccordance with the present invention are maintained in such amulti-layer composite.

[0011] Further objects, features, and advantages of the invention willbe apparent from the following detailed description taken in conjunctionwith the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012]FIG. 1 is a flowchart diagram of an exemplary method for creatinga two-layer rendered scene image in accordance with the presentinvention.

[0013]FIG. 2 is an exemplary background image layer scene to becomposited with a foreground layer scene rendered in accordance with thepresent invention.

[0014]FIG. 3 is an exemplary foreground image layer scene, with thebackground of the foreground image layer erased to full black, to becomposited with the background image layer of FIG. 2.

[0015]FIG. 4 is an exemplary image scene resulting from the compositionof the background image layer of FIG. 2 with the foreground image layerof FIG. 3, rendered in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0016] An exemplary method 10 of creating a rendered multi-layer sceneimage in accordance with the present invention will be described indetail with reference to the flowchart diagram of FIG. 1. The exemplarymethod 10 to be described in detail herein features the compositing oftwo RGB image layers, a background image layer and a foreground imagelayer. Each of the background and foreground image layers are renderedseparately, and may be saved and composited at a later time to form acomposition which will appear the same as if it were rendered all at thesame time. It should be understood that the present invention may beemployed very effectively for the creation of rendered scenes that arecomposed of more than two RGB image layers. For example, the presentinvention may be employed effectively to render the foreground imagelayer for a multi-layer virtual presentation incorporatingthree-dimensional graphics or animations with a live or pre-recordedtwo-dimensional video image positioned between foreground and backgroundlayers of the three-dimensional image. Such a Virtual PresentationSystem and Method with which the present invention may be employed isdescribed in U.S. patent application Ser. No. 09/435,552, filed Nov. 8,1999, by Richard T. Daly, et al., which application is assigned to theassignee of the present application. Based on the detailed descriptionpresented below, a computer programmer of ordinary skill in the art ofcomputer graphics and image processing will be able to implement alayered image rendering method in accordance with the present inventionon a conventional computer system.

[0017] In accordance with the present invention, background andforeground image layers to be composited are rendered separately inRGBA. An RGBA background image layer is rendered at 12. The backgroundimage layer may be rendered in a conventional manner. The backgroundimage layer scene may include computer graphics generated images, and/oranimated or still digitized photographic images, etc. The RGBAbackground image layer scene thus rendered is saved at 14, in aconventional manner, to be composited with a foreground image layerscene at a later time. Note that the background layer may be rendered asan RGB image with no alpha. If the background layer is rendered as anRGBA image, the alpha in the background layer may be used duringcompositing in a conventional manner, e.g., as a holdback mat. Anexemplary background image layer scene 16, created in a conventionalmanner, is illustrated in FIG. 2. The exemplary background image layerscene 16 is a digital satellite image of North America.

[0018] A foreground image layer is created at 18 by first defining theforeground image layer scene in a conventional manner. The elements,objects, or components appearing in the foreground image layer may bedefined and created using any conventional computer graphics generationtechnique. An exemplary foreground image layer scene 20 is shown in FIG.3. The exemplary foreground image layer scene 20 is a graphicalrepresentation of the state borders of the United States.

[0019] The background of the foreground image layer is erased to fullblack at step 22. In other words, the foreground image layer sceneelements are defined over a full black background. (The exemplaryforeground image layer elements 20 are shown in FIG. 3 with such a fullblack background.)

[0020] The elements of the foreground image layer scene over the blackbackground are rendered in a conventional manner at step 26 to obtainRGB components for each pixel of the foreground image layer scenerendered over black. We will refer to the resulting RGB image layerscene as FLB (foreground layer rendered over black).

[0021] At step 28 the background of the foreground image layer is erasedto full white. In other words, the elements of the foreground imagelayer scene are defined over a full white background.

[0022] At step 30 the elements of the foreground image layer scene overthe white background are rendered in a conventional manner to obtain RGBcomponents for each pixel of the foreground image layer scene renderedover white. We will refer to this rendered image as FLW (foregroundlayer rendered over white). (It should be noted that the sequence ofsteps 22, 26 and 28, 30 may be performed in any order, orsimultaneously.)

[0023] The RGB components for each pixel of the foreground image layerscene rendered over black (FLB) are then combined with the RGBcomponents for each corresponding pixel of the foreground image layerscene rendered over white (FLW) at 32 to form the rendered foregroundimage layer (FL). A description of the preferred procedure for formingthe foreground image layer (FL) from the foreground layer scene renderedover black (FLB) and the foreground layer scene rendered over white(FLW) follows.

[0024] For each corresponding pixel of the foreground image layerrendered over black (FLB) and white (FLW), a new RGBA foreground imagelayer (FL) pixel is created. The alpha value for each pixel of the newforeground image layer (FL) is determined as one plus the value of acolor component (R, G, or B) of the corresponding pixel from theforeground image layer rendered over black (FLB) minus the value of thesame color component of the corresponding pixel from the foregroundimage layer rendered over white (FLW). For example, an alpha value for apixel of the new foreground layer (FL) may be determined as one plus thevalue of the red (R) component of the corresponding pixel from theforeground image layer rendered over black (FLB) minus the value of thered (R) component of the corresponding pixel from the foreground imagelayer rendered over white (FLW).

[0025] If the resulting alpha value of the pixel is zero, the value ofall of the RGB color components of the pixel are set to zero. Otherwise,the RGB color component values of the pixel of the new foreground layer(FL) are set to the corresponding color component values (R, G, and B)of the corresponding pixel from the foreground image layer scenerendered over black (FLB) divided by the alpha value for the pixel. Thisprocedure is repeated for each pixel of the foreground layer (FL) to berendered. The resulting foreground layer (FL) is saved at 34, to becombined with the background layer image, or other image layers.

[0026] The procedure described above for rendering the foreground layer(FL) may be summarized by the following exemplary equations which areapplied to generate each pixel of the foreground layer (FL):

(FL alpha)=1+(FLB red component)−(FLW red component)

[0027] If (FL alpha)=0, then

[0028] (FL red )=0

[0029] (FL green)=0

[0030] (FL blue)=0

[0031] Else

(FL red)=(FLB red component)/(FL alpha)

(FL green)=(FLB green component)/(FL alpha)

(FL blue)=(FLB blue component)/(FL alpha)

[0032] Thus, the present invention may be employed in combination withconventional RGBA rendering techniques and programs to derive colorcomponents and alpha values for an RGBA foreground layer image to becomposited with a background layer image. The resulting renderedforeground layer (FL) may be combined with the saved background layer,using conventional compositing techniques, to form the composition imagescene at 36. An exemplary composition 38 of the background image layer16 of FIG. 2 with the foreground image layer 20 of FIG. 3, rendered inaccordance with the present invention, is shown in FIG. 4. The resultingcomposition appears the same as if the entire image were rendered all atthe same time, rather than resulting from the composition of separatelyrendered background and foreground image layers. A foreground imagelayer rendered in accordance with the present invention may be combinedwith any background layer image, or more than one background orintermediate layers, without adversely affecting the quality of theforeground layer as appearing in the composition image.

[0033] It should be understood that the present invention is not limitedto the exemplary applications and embodiments illustrated and describedherein, but embraces such modified forms thereof as come within thescope of the following claims. In particular, it should be understoodthat a method in accordance with the present invention is not limited tothe particular sequence of steps and equations described above.

What is claimed is:
 1. A method for rendering an image layer scene,comprising the steps of: (a) defining a scene of image layer elements;(b) rendering the elements of the image layer scene over a blackbackground to obtain RGB components for each pixel of the image layerscene rendered over black; (c) rendering the elements of the image layerscene over a white background to obtain RGB components for each pixel ofthe image layer scene rendered over white; and (d) combining the RGBcomponents for each pixel of the image layer scene rendered over blackwith the RGB components for each corresponding pixel of the image layerscene rendered over white to form the rendered image layer scene.
 2. Themethod of claim 1 wherein the step of combining the RGB components foreach pixel of the image layer scene rendered over black with the RGBcomponents for each corresponding pixel of the image layer scenerendered over white includes the steps of, for each corresponding pixelof the image layer scenes rendered over black and white: (a) determiningan alpha value for the pixel as one plus the value of a color componentof the pixel from the image layer scene rendered over black minus thevalue of the same color component of the corresponding pixel from theimage layer scene rendered over white; (b) setting all of the RGB colorcomponent values of the pixel to zero if the alpha value for the pixelequals zero; (c) otherwise setting the RGB color component values of thepixel to the corresponding color component values of the correspondingpixel from the image layer scene rendered over black divided by thealpha value for the pixel.
 3. The method of claim 2 wherein the step ofdetermining an alpha value for the pixel includes the step ofdetermining the alpha value for the pixel as one plus the value of a redcomponent of the pixel from the image layer scene rendered over blackminus the value of the red component of the corresponding pixel from theimage layer scene rendered over white.
 4. A method for rendering amulti-layer image, comprising the steps of: (a) rendering a backgroundimage layer; (b) saving the background image layer; (c) creating aforeground image layer scene of foreground image layer elements; (d)rendering the elements of the foreground image layer scene over a blackbackground to obtain RGB components for each pixel of the foregroundimage layer scene rendered over black; (e) rendering the elements of theforeground image layer scene over a white background to obtain RGBcomponents for each pixel of the foreground image layer scene renderedover white; (f) combining the RGB components for each pixel of theforeground image layer scene rendered over black with the RGB componentsfor each corresponding pixel of the foreground image layer scenerendered over white to form a rendered foreground image layer; and (g)compositing the background image layer and the foreground image layer toform a multi-layer image.
 5. The method of claim 4 wherein the step ofcombining the RGB components for each pixel of the foreground imagelayer scene rendered over black with the RGB components for eachcorresponding pixel of the foreground image layer scene rendered overwhite includes the steps of, for each corresponding pixel of theforeground image layer scenes rendered over black and white: (a)determining an alpha value for the pixel as one plus the value of acolor component of the pixel from the foreground image layer scenerendered over black minus the value of the same color component of thecorresponding pixel from the foreground image layer scene rendered overwhite; (b) setting all of the RGB color component values of the pixel tozero if the alpha value for the pixel equals zero; (c) otherwise settingthe RGB color component values of the pixel to the corresponding colorcomponent values of the corresponding pixel from the foreground imagelayer scene rendered over black divided by the alpha value for thepixel.
 6. The method of claim 5 wherein the step of determining an alphavalue for the pixel includes the step of determining the alpha value forthe pixel as one plus the value of a red component of the pixel from theforeground image layer scene rendered over black minus the value of thered component of the corresponding pixel from the foreground image layerscene rendered over white.
 7. The method of claim 4 comprisingadditionally the steps of providing a third image layer and compositingthe background image layer, the foreground image layer, and the thirdimage layer to form a multi-layer image with the third image layerappearing between the background image layer and the foreground imagelayer in the composited multi-layer image.
 8. The method of claim 4wherein the step of rendering a background image layer includes the stepof rendering an RGB background image layer.